Expanding the utility of the ROX index among patients with acute hypoxemic respiratory failure

Background Delaying intubation in patients who fail high-flow nasal cannula (HFNC) may result in increased mortality. The ROX index has been validated to predict HFNC failure among pneumonia patients with acute hypoxemic respiratory failure (AHRF), but little information is available for non-pneumonia causes. In this study, we validate the ROX index among AHRF patients due to both pneumonia or non-pneumonia causes, focusing on early prediction. Methods This was a retrospective observational study in eight Singapore intensive care units from 1 January 2015 to 30 September 2017. All patients >18 years who were treated with HFNC for AHRF were eligible and recruited. Clinical parameters and arterial blood gas values at HFNC initiation and one hour were recorded. HFNC failure was defined as requiring intubation post-HFNC initiation. Results HFNC was used in 483 patients with 185 (38.3%) failing HFNC. Among pneumonia patients, the ROX index was most discriminatory in pneumonia patients one hour after HFNC initiation [AUC 0.71 (95% CI 0.64–0.79)], with a threshold value of <6.06 at one hour predicting HFNC failure (sensitivity 51%, specificity 80%, positive predictive value 61%, negative predictive value 73%). The discriminatory power remained moderate among pneumonia patients upon HFNC initiation [AUC 0.65 (95% CI 0.57–0.72)], non-pneumonia patients at HFNC initiation [AUC 0.62 (95% CI 0.55–0.69)] and one hour later [AUC 0.63 (95% CI 0.56–0.70)]. Conclusion The ROX index demonstrated moderate discriminatory power among patients with either pneumonia or non-pneumonia-related AHRF at HFNC initiation and one hour later.

Although HFNC has been used among AHRF patients with non-pneumonia indications, existing evidence for HFNC use has been mainly among pneumonia patients. The FLORALI study demonstrated lower intubation rates among pneumonia patients with PaO 2 :FiO 2 ratio (PF ratio) less than 200 mmHg when compared to those receiving conventional oxygen therapy (COT) and non-invasive ventilation (NIV), although up to 40% still required intubation [7]. Prior studies among non-pneumonia patients focused mainly on physiological parameters [8][9][10][11][12][13][14], with the failure rate remaining largely unknown. Understanding HFNC failure rates is important, because mortality may be increased if intubation is delayed [15].
In this respect, Roca and colleagues developed the ROX index to help predict HFNC failure in patients with pneumonia [16,17]. The ROX index is the ratio of SpO 2 /FiO 2 (SF ratio) to respiratory rate. It has excellent specificity (98-99%) for predicting HFNC failure at various time points of 2 (<2.85), 6 (<3.47) and 12 hours (<3.85), albeit at the expense of low sensitivity [17]. Physicians have subsequently attempted to improve the ROX index [18] and explore its utility as a predictor of weaning failure [19]. Current evidence supporting the use of the ROX index is limited to pneumonia patients mainly at later time points, limiting its utility.
Understanding the performance of the ROX index at earlier time points, would enhance prediction of HFNC failure, facilitating earlier intubation and potentially averting increased mortality risks. In this study, we aimed to expand the utility of the ROX index as a tool for predicting HFNC failure by validating its use at earlier time points among both pneumonia and non-pneumonia patients. We hypothesized that the ROX index could discriminate patients likely to experience HFNC failure at earlier time points in both pneumonia and non-pneumonia AHRF.

Study design
We conducted a retrospective observational multi-center study in all public hospital intensive care units (ICU) in Singapore of patients using HFNC for AHRF from 1 January 2015 to 30 September 2017. The National Healthcare Group Domain-Specific Review Board approved the study with a waiver of informed consent due to the non-interventional study design (DSRB 2017/00900).

Patients
Patients were included if they were older than 18 years and treated with HFNC for AHRF. Patients with concomitant hypercapnia were also included. Patients were excluded if they had do-not-intubate orders for the current AHRF episode or if HFNC was used for pre-oxygenation, post-extubation management or peri-procedural purposes. HFNC initiation was at the discretion of the primary physician.

Data collection
Demographic characteristics were collected for all patients including: age, gender, ethnicity, admission source and comorbidities (diabetes, hypertension, ischemic heart disease, stroke, asthma, COPD, bronchiectasis, interstitial lung disease, liver cirrhosis and chronic kidney disease). Immunocompromised state was also determined, defined as the presence of any solid tumor, hematological malignancy, or usage of steroids (at least 0.3mg/kg/day for at least 1 month) [20] or other immunosuppressants. Illness severity was determined from the Acute Physiology and Chronic Health Evaluation (APACHE) II score and frequency of vasopressor use. The underlying etiology of AHRF was categorized into one of the following: (a) pneumonia, defined as the presence of respiratory symptoms with radiological evidence on chest radiograph; (b) heart failure; (c) COPD exacerbation; (d) asthma exacerbation; (e) exacerbation of interstitial lung disease; (f) HFNC use post-surgery; (g) others. In cases where multiple etiologies may account for the patient's AHRF, the leading etiology was determined by the primary physician.
HFNC failure was defined as requiring intubation after HFNC initiation. However, since this was a retrospective observational study, and none of the participating ICUs used a specific protocol to determine intubation decisions among the HFNC patients, the intubation decision occurred at the discretion of the managing physician. ICU mortality, hospital mortality, ICU length-of-stay and hospital length-of-stay were also collected.
As mentioned, the ROX index is the ratio of the SF ratio to respiratory rate. In order to compute the ROX index as defined by Roca and colleagues [17], the following parameters were collected immediately prior to HFNC initiation, after one hour and in the event of HFNC failure, at the point of intubation: SpO 2 (%), FiO 2 (%), respiratory rate (breaths/min), flow rate (L/min) and arterial partial pressure of carbon dioxide (PaCO 2 , in mmHg) [16,17].

Statistical analysis
Univariate comparisons of proportions, means and medians were performed using the Chisquare test, Student t, and Wilcoxon rank-sum tests respectively. Separate subgroup analyses were performed for both pneumonia and non-pneumonia populations. We assessed the discrimination of the ROX index using receiving operating characteristic (ROC) curves. Using parameters at HFNC initiation and one hour later, the Youden index method was used to determine the optimal cut-point of the ROX index for HFNC failure. Statistical significance was taken as a two-tailed P<0.05, and analyses were performed with Stata 15.0 (College Station, TX).

Patient characteristics and outcomes
Eight ICUs participated in this study. 483 patients required HFNC for AHRF, of whom 45 patients also had concomitant hypercapnia. 263 patients (54%) had a primary diagnosis of pneumonia (Fig 1). The remaining patients who received HFNC had cardiovascular conditions (n = 58), utilized HFNC post-surgery (n = 57), and for non-pulmonary acute respiratory distress syndrome (n = 25), COPD (n = 14), interstitial lung disease (n = 7) or asthma exacerbations (n = 1). 185 patients (38.3%) failed HFNC. Baseline demographics were mostly similar between both patients with pneumonia and non-pneumonia conditions. Patients with nonpneumonia conditions were more likely to have ischemic heart disease and less likely to be admitted from the operating theatre (Table 1).

Discussion
Among AHRF patients with pneumonia, the ROX index demonstrated moderate discriminatory power to predict HFNC failure, with greater discrimination one-hour post-initiation compared to when HFNC was initiated. In a similar fashion, the ROX index was also validated as moderately discriminating among non-pneumonia AHRF patients at similar time points. Roca and colleagues originally validated the ROX index in patients with pneumonia [16,17]. However, at least 20% of all ICU patients with AHRF suffer from non-pneumoniarelated conditions [21]. In some subgroups such as immunocompromised patients, the etiology of AHRF is even more varied [20,[22][23][24][25], and in ED, HFNC may be initiated before the underlying etiology of AHRF is established [9]. Since delayed intubation can increase mortality regardless of the cause of AHRF [7], it is important to understand the performance of the ROX index among patients with non-pneumonia-related conditions. Currently, there are few studies investigating HFNC use among this population and they mainly focused on demonstrating physiological benefits [2,7,[26][27][28]. In contrast, our study focused on understanding the performance of the ROX index among a more diverse HFNC population.
In addition, as the ROX index was originally validated at 2, 6 and 12 hours [16,17], we explored its ability to discriminate patients likely to fail HFNC at earlier time points. This would facilitate early pre-emptive intubation before further deterioration has occurred [15]. We demonstrated that at one-hour post-HFNC initiation, the AUROC of the ROX index for predicting HFNC failure was 0.71 and 0.63 for pneumonia and non-pneumonia patients respectively. Therefore, we have shown that the ROX index calculated immediately prior to HFNC initiation and one-hour post-initiation can help identify those likely to fail. Since HFNC is increasingly initiated while patients are still in ED, predicting HFNC failure at earlier time points can guide the development of practical workflows for close monitoring, especially in settings where staffing resources for close patient monitoring is scarce, or transfer time from ED to ICU is prolonged. Identifying patients most likely to fail HFNC may facilitate earlier intubation decisions and increase safety during patient transport [29].
It is important to highlight that the ROX index is a screening tool with only moderate discriminatory power. In the original study, Roca and colleagues report AUROC values at 2 hours of 0.679, 6 hours of 0.703 and 12 hours of 0.752 [17]. We report similar values in our study (Table 3). To maximize the screening tool potential of the ROX index, the values adopted favoured higher specificity and negative predictive values, so as to reduce false positives. To illustrate, a ROX index of <6.06 (sensitivity 51%, specificity 80%) among pneumonia patients one-hour post-HFNC initiation would have led to an additional 46 patients being intubated. Importantly, our study also highlights the importance of serial measurements of the ROX index given the dynamic evolution of the disease course among these patients. Depending on the study population, the exact ROX index value predicting HFNC failure may differ. This is illustrated with the studies by Roca and Goh separately depicting different values at the same time points at 2 (<2.85 vs <6.55), 6 (<3.47 vs <6.60) and 12 (<3.85 vs <6.55) hours [17,18].
Our study has several strengths. It is a practical real-world evaluation. Our study population was large, from multiple centers and included AHRF patients with multiple etiologies. This allowed us to validate the ROX index at earlier time points and among non-pneumonia patients. To date, this is the largest study population investigating HFNC failure among nonpneumonia patients. However, we acknowledge several limitations in our study. Firstly, it was conducted retrospectively, and in the absence of a standardized protocol, we were unable to study the ROX index at other time points, in particular at later time points, which could be a subject for future studies. Secondly, the lack of a standardized protocol for HFNC practice could have led to biases from individual physician practice patterns and thresholds for determining HFNC failure. Thirdly, as with all retrospective studies, some missing data is inevitable, with the majority related to arterial blood gas findings one-hour post-HFNC initiation. Other studies have already established that these parameters do not predict HFNC failure [16,18,19]. Otherwise, all other parameters were missing <10% of the total data, thus mitigating the bias risk (S1 Table). Fourthly, determination of HFNC failure was clinically-driven rather than protocolized, and some variation of HFNC failure thresholds existed across centers (S3 and S4 Tables). Importantly, our HFNC failure rate of 38.3% is also similar to that of the FLORALI study [7]. Finally, in the absence of randomization, we are unable to account for the effect of unmeasured variables that may affect the ROX index.

Conclusion
There is moderate discriminatory power of the ROX index to predict HFNC failure among both pneumonia and non-pneumonia patients with AHRF upon HFNC initiation and onehour post-initiation. Project administration: Andrew Li, Jason Phua.